Skip to content

feat: derive heredoc separators from content hash for build stability#37224

Merged
pelikhan merged 2 commits into
mainfrom
copilot/update-compiler-algorithm
Jun 6, 2026
Merged

feat: derive heredoc separators from content hash for build stability#37224
pelikhan merged 2 commits into
mainfrom
copilot/update-compiler-algorithm

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jun 6, 2026

Here-doc separators were previously derived from an HMAC of the workflow frontmatter hash, causing all separators to change whenever any unrelated frontmatter changed. Separators now hash the content they wrap (SHA-256), so they change only when their own content changes — reducing diff noise and merge conflicts.

Core change

New function in strings.go:

// GenerateHeredocDelimiterFromContent derives a stable delimiter from SHA-256(UPPER(name) || content).
// The name prefix ensures two heredocs wrapping identical content still produce distinct delimiters.
func GenerateHeredocDelimiterFromContent(name string, content string) string {
    h := sha256.New()
    h.Write([]byte(strings.ToUpper(name)))
    h.Write([]byte(content))
    tag := hex.EncodeToString(h.Sum(nil)[:8])
    ...
}

Output format (GH_AW_<NAME>_<16hex>_EOF) is unchanged, so all existing validation and normalization code works as-is.

Call site changes

  • mcp_renderer.go, mcp_setup_generator.go, codex_mcp.go (custom config) — content variable was already in scope; one-line swap.
  • compiler_safe_outputs_job.go — reordered to generate scriptContent before the delimiter; removed now-unused frontmatterHash parameter from buildCustomScriptFilesStep.
  • codex_mcp.go (MCP_CONFIG, shell policy) — content was built incrementally; refactored to accumulate into a strings.Builder, hash, then write to YAML output.
  • unified_prompt_step.go — hashes the concatenation of all built-in section content and user prompt chunks, which is the stable compile-time representation of the prompt.

Tests

Six new unit tests covering stability, content-sensitivity, name disambiguation, empty inputs, and case normalisation. Golden fixtures updated to reflect the new stable delimiter values (e.g. GH_AW_PROMPT_2db13b3f20a79b91_EOF).

GenerateHeredocDelimiterFromSeed is retained for backward compatibility.

Copilot AI and others added 2 commits June 6, 2026 00:15
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title feat: generate heredoc separators from content hash for build stability feat: derive heredoc separators from content hash for build stability Jun 6, 2026
Copilot AI requested a review from pelikhan June 6, 2026 00:17
@pelikhan pelikhan marked this pull request as ready for review June 6, 2026 01:03
Copilot AI review requested due to automatic review settings June 6, 2026 01:03
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates heredoc delimiter generation across the workflow compiler to reduce diff noise by deriving delimiters from the content they wrap (instead of a workflow/frontmatter-derived seed), and updates unit tests + golden fixtures accordingly.

Changes:

  • Added GenerateHeredocDelimiterFromContent(name, content) (SHA-256-based) and updated multiple call sites to use it.
  • Refactored some generators to compute content first (so delimiters can be derived from that content) and adjusted tests to match new function signatures.
  • Updated golden fixtures to reflect newly generated outputs.
Show a summary per file
File Description
pkg/workflow/strings.go Introduces content-derived heredoc delimiter helper.
pkg/workflow/strings_test.go Adds unit tests for the new content-derived delimiter behavior.
pkg/workflow/unified_prompt_step.go Switches prompt heredoc delimiter derivation to be content-based.
pkg/workflow/mcp_renderer.go Switches MCP JSON config heredoc delimiter derivation to be content-based.
pkg/workflow/mcp_setup_generator.go Switches multiple heredoc delimiters (safe-outputs + mcp-scripts) to be content-based.
pkg/workflow/compiler_safe_outputs_job.go Refactors safe-output script step generation to derive delimiter from script content.
pkg/workflow/codex_mcp.go Refactors TOML heredoc generation to buffer content then derive a content-based delimiter.
pkg/workflow/safe_scripts_test.go Updates tests for the buildCustomScriptFilesStep signature change.
pkg/workflow/testdata/TestWasmGolden_CompileFixtures/*.golden Updates compiled output fixtures (including engine/CLI version lines).
pkg/workflow/testdata/TestWasmGolden_AllEngines/*.golden Updates compiled output fixtures (including engine/CLI version lines).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 15/15 changed files
  • Comments generated: 3

Comment thread pkg/workflow/strings.go
Comment on lines +193 to +199
// GenerateHeredocDelimiterFromContent creates a stable heredoc delimiter derived from the
// content it wraps. The 16-character hex tag is the first 8 bytes of SHA-256 of the content,
// so the delimiter is identical across builds whenever the content is unchanged and changes
// only when the content changes — reducing unnecessary diff noise and merge conflicts.
//
// The name prefix (e.g. "PROMPT", "MCP_CONFIG") is included for readability and to ensure
// that two different heredocs wrapping identical content still produce distinct delimiters.
Comment on lines +270 to +279
// Derive the heredoc delimiter from the combined prompt content so it is identical
// across builds for the same workflow and changes only when the prompt text changes.
var promptContentForHash strings.Builder
for _, section := range builtinSections {
promptContentForHash.WriteString(section.Content)
}
for _, chunk := range userPromptChunks {
promptContentForHash.WriteString(chunk)
}
delimiter := GenerateHeredocDelimiterFromContent("PROMPT", promptContentForHash.String())
Comment on lines 59 to +62
env:
GH_AW_SETUP_WORKFLOW_NAME: "with-imports-test"
GH_AW_CURRENT_WORKFLOW_REF: ${{ github.repository }}/.github/workflows/with-imports.lock.yml@${{ github.ref }}
GH_AW_INFO_VERSION: "1.0.57"
GH_AW_INFO_VERSION: "1.0.59"
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 6, 2026

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 6, 2026

Design Decision Gate 🏗️ failed during design decision gate check.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 6, 2026

Test Quality Sentinel failed during test quality analysis.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 6, 2026

⚠️ PR Code Quality Reviewer failed during code quality review.

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Jun 6, 2026

@pelikhan pelikhan merged commit e29e862 into main Jun 6, 2026
83 of 90 checks passed
@pelikhan pelikhan deleted the copilot/update-compiler-algorithm branch June 6, 2026 01:23
Copilot stopped work on behalf of pelikhan due to an error June 6, 2026 01:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 6, 2026

@copilot review all comments and address the lint-go failure, then rerun CI.

Generated by 👨‍🍳 PR Sous Chef · agent 45.4 AIC · threat-detection 4.79 AIC ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants